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ABSTRACT 


In order to meet the requirements of engineering education professional certification, to ensure the engineering teaching quality and promote the teaching ability, the 
grading method of courses based on engineering professional certification is preliminary explored. The statistic method of analyzing the scores for achievement is 
applied for the proper evaluation of the test questions. The results show that the traditional methods of evaluation exit shortcomings. A comprehensive method to 
evaluate the students' course tests by combining the indexes of difficulty, distinction, effectiveness, feasibility and score distribution are proposed, a typical example is 


provided as an illustration. 
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1. INTRODUCTION: 

Examination plays an important role in the teaching process, and its main pur- 
pose is to evaluate the degree and level of the knowledge to learn, understand and 
master for students, as well as an important method to test the teaching quality 
and existing problems for teachers, which can be used to improve the teaching 
methods, teaching quality and promote the development the students’ quality. 
Grade evaluation is the baton of the teaching process, which not only commands 
the teacher's "teaching", but also commands the students' "learning", which plays 
a vital role in the teaching effect and the quality of talent training. With the devel- 
opment of the project education professional certification, the professional cer- 
tification has put forward the specific requirements for the student academic per- 
formance evaluation. Therefore, in the context of professional certification, it is 
of great significance to establish a reasonable evaluation system for course per- 
formance combined with the actual situation of the school. In this paper, based on 
the problems in the grade evaluation of traditional course, the mathematical sta- 
tistics method is used to quantitatively calculate the difficulty, differentiation, 
test paper feasibility and effectiveness, so as to analyze and evaluate the test 
paper objectively, accurately and quantitatively, which can more effectively test 
the students' learning level and the teacher's teaching effect, which also truly 
reflect the completion of the teaching program and the achievement of the course 
objectives [1-4]. At the same time, taking into account the fairness, rationality 
and operability of the evaluation, in this paper, the method of "comprehensive 
examination of difficulty, differentiation, effectiveness, feasibility and score dis- 
tribution" was put forward to evaluate students' course score. In our school, the 
test score analysis of curriculum includes score at ordinary times, final term 
exam grade and comprehensive curriculum grade. they all were analyzed in 
degree of difficulty, differentiation, feasibility and effectiveness. 


2. GRADING METHOD OF COURSES: 

2.1 Difficulty analysis (P): 

Difficulty is the degree of difficulty in examining test questions or test papers, 
which is one of the main indexes to evaluate the quality of examination papers. 


(1) Calculated method: 
Assume: P is the difficulty factor of the test paper, X is the average score of 
the test paper, and W is the full score of the test paper, then the difficulty of 
the test paper is: 





(1) 


Suppose: P, is the difficulty factor of question jin the test paper, X, is the aver- 
age score of the question / for all students, W, is the full score of the question 
i, then the difficulty of question I is: 
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P}=1- (2) 
i 
(2) Criteria of judgement: 
The closer the difficulty factor is tend to 0, the smaller the difficulty; the 
closer the difficulty factor is tend to 1, the greater the difficulty. 


When P¥0.2, the difficulty is too low; 


When 0.2 <P<0.4, the difficulty is moderate; 
When 0.4 <P<0.8, it means that it is more difficult; 
When P> 0.8, the difficulty is too high. 


2.2 Differentiation analysis (D): 

Differentiation is the degree of differentiation or identification ability of exami- 
nation questions or test papers to the students' actual level of knowledge. Exami- 
nation questions or papers, with high degree of differentiation, which can distin- 
guish students with different knowledge levels and abilities, so that students with 
higher abilities have higher scores, and students with lower abilities have lower 
scores; test questions or test papers of low differentiation can not distinguish the 
abilities and levels of knowledge of the students. 


(1) Calculated method: 
Firstly, the students' test scores of question i are sequenced in order from 
high to low, taking the top 27% of the students as high grade group, while the 
bottom 27% are selected as the low grade group. Students' average scores of 
the question i in the high and low grade group are calculated as X,, and X, 
respectively. 


Ifthe full score of question 1 is W,, then differentiation of question is 


xX 
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Similarly, the average grade of the top 27% of the high-grade group is set as 
X,, the average grade of the bottom 27% of the low- grade group is set as X,, 
and full score of the test paper is W, then differentiation of the test paper is 


G=—4 i (4) 
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Criteria of judgement: 
When D>0.4, the differentiation is “excellent”; 


When 0.3<DS0.4, the differentiation is "good"; 

When 0.2 <D < 0.3, the differentiation is "OK"; 

When D<0.2, the differentiation is “poor”. 
2.2 Feasibility analysis(B): 
Feasibility is an indicator to measure the feasibility and stability of test questions. 
High feasibility indicates that students' scores are not easily affected by the errors 
caused by accidental factors, and the candidate's score can truly reflect the actual 


level of students. 


(1) Calculation formula: 
The feasibility B of the examination paper is: 
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n-1| . = (5) 








Among them, n represents the total number of questions, Si indicates the 
standard deviation of question i, and S, represents the variance of test papers. 


(6) 


Among them, m indicates the number of students, and X, indicates the score 
of question i of j student, indicates the average score of all students' ques- 
tions i. 
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Among them, m is the number of students and X; is the total score of test 
paper of 7 student, indicates the average score of test papers of all students. 


(2) Criteria of judgement: 
When B<0.5, the feasibility of test questions is “poor”; 


When 0.5<B<0.8, the feasibility of test questions is “good”; 
When B>0.8, the feasibility of test questions is “very good”. 


2.4 Effectiveness analysis (R): 

Effectiveness is an index to measure the degree of achievement of test objectives, 
which reflects the accuracy and effectiveness of the test. In order to improve the 
effectiveness of the examination, two kinds of questions should be paid atten- 
tion: firstly, the goal of the examination should be clear, it is to test the ability to 
master basic knowledge for students, or to test the students' ability to apply 
knowledge for judgment, or a combination of both; secondly, the design of the 
test questions should be able to effectively reflect the test objectives, in which the 
objective test questions are generally used to assess students' mastery of basic 
knowledge, while the non-objective test questions are used to assess students' 
ability to apply knowledge for judgment. 


(1) Calculation formula: 
The effectiveness of the test paper can be measured by the average differen- 
tiation of the test paper, thus: 


R= i=l 
i? 


(8) 


Among them, R is the effectiveness of the test paper, D, is differentiation of 
question i, and is the total number of test questions. 


(2) Criteria of judgement: 
When R>0.4, the effectiveness of the test paper is good, that is, the goal of 
the assessment can be effectively achieved. 


Taking the course of pharmaceutical separation engineering in our school as 
an example, the evaluation result of examination paper is illustrated, which 
includes the analysis of test score and test quality. Analysis of test score also 
includes the problems found and the improvement actions except the aver- 
age score and frequency distribution chart. The quality analysis of the test 
questions includes the difficulty, differentiation, feasibility and effective- 
ness. 


(1) Evaluation of test results: 

In this test, the highest score is 96, the lowest score is 45, and the average 
score is 76.8, of which 2 students failed the final exam, and the pass rate 
reaches 96.4%. The percentage distribution of each fraction is shown in Fig- 
ure 1, and the distribution of the test results is shown in Figure 2. There is a 
51-point gap between the highest score and the lowest score, which shows 
that the students with good learning ability and poor learning ability have 
great differences in their grades. Teachers should pay more attention to the 
students with poor learning ability in future teaching. 
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79.5-85.4 12.4. 
64.5-69.4 B.A, 
85.5-89.4 BA, 
89.5-100.0 8A, 
59.5-64.4 7A, 
74.5-79.4 6A, 
0.0-59.4 4A, 
695-744 4A, 
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Figure 1: Percentage distribution of each fraction; 


O 5 11°16 21 26 32 3/ 42 47 53 58 63 68 7/4 79 84 89 95 


Figure 2: Distribution of test scores 





(2) Evaluating the quality of test papers: 
The total score of this paper is 100, which includes 3 Question Type, that is 
filling blanks (50 questions, 50 points), discussion questions (6 questions, 
30 points), design questions (2 questions, 20 points), respectively, and the 
proportion of each type is shown in Table 1. 


Table 1: Distribution of test questions in pharmaceutical separation 
engineering 
| = 


SUA oo) Score | Percentage 
| Question | | 8 


Question types 














Objective question Filling blanks 50 50 50% 
ee ___ Discussion questions 6 30 30% | 
Subjective question t 
Design questions 2 20 20% | 
Total 58 100 100% 























In this test, the content of the questions 1s relatively extensive, which overwrites 
most of all existing difficulties and key points in the textbooks, which can not 
only test the degree of master knowledge, but also evaluate the comprehensive 
understanding and application of knowledge for students. 


As an important parameter for evaluating the quality of test papers, such as diffi- 
culty analysis, differentiation analysis, effectiveness analysis, feasibility analy- 
sis, which is also an important basis for evaluating the effectiveness and feasibil- 
ity of score. Overall, the average difficulty of this test paper is 0.44, which is con- 
sistent with the other test papers of 0.40 - 0.80 of P value; the differentiation is 
0.46, indicating that the quality of the test paper is good; the feasibility is 0.6875, 
which reflects the size of the random error in the test; the validity is 0.5236, 
which also reflects the size of system error in the test. 


3. CONCLUSION: 

Engineering education professional certification puts forward higher require- 
ments for the test evaluation system. Effective test analysis can objectively 
reflect the teaching level and effect, which can help teachers and students to find 
teaching bottleneck problem, improving our teaching method and teaching qual- 
ity. An exam is a measurement, in order to improve the accuracy of measurement, 
it is necessary to use scientific measurement theory, appropriate measurement 
tools and scientific quantitative evaluation index to analyze the test paper. Any 
method of course evaluation has its advantages and disadvantages. Only based 
on the full understanding of various methods, our own advantages can be kept in 
the course evaluation reform. By learning from the experience of universities at 
home and abroad rationally, learning from other's strong points to make up our 


International Education & Research Journal [[ERJ] 





E-ISSN No : 2454-9916 | Volume: 4 | Issue: 11 | Nov 2018 


deficiencies, our grading method of courses can be beneficial to promote com- 
prehensive and the well-rounded development of student. 
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